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DETAILED ACTION 

This Office Action is in response to the amendment filed March 21, 2008. 



Claim Rejections - 35 USC § 101 
1. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 21-37 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non- 
statutory subject matter. Claims 21-37 are directed to computer program product embodied in a 
carrier. A signal is not a statutory class of invention under 35 U.S.C 101. Therefore, the claims 
fail to be limited only to embodiments which fall within a statutory class of invention. 



Claim Rejections - 35 USC §102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of appUcation for patent in the United States. 

3. Claims 1-46 are rejected under 35 U.S.C. 102(b) as being anticipated by Riis et al, 
"Multilingual Text-to-Phoneme Mapping," Proceedings of the Eurospeech 2001 Conference. 

4. Regarding claim 1, Riis discloses accepting text spellings (Figure 2, "text input") of 
training words in a plurality of sets of training words, each set corresponding to a different one of 
a plurality of languages (section 3, Multilingual TTP Mapping); for each of the sets of training 
words in the plurality, receiving pronunciations for the training words in the set, the 
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pronunciations being characteristic of native speakers of the language of the set, the 
pronunciations also being in terms of subword units at least some of which are common to two 
or more of the languages (Figure 3); training a single pronunciation estimator using data 
comprising the text spellings and the pronunciations of the training words (Figure 2, 
"Multilingual Recognizer"); and calculating an acoustic subword model for each subword unit, 
based on the pronunciations in the plurality of sets of training words, by mixing distributions of 
acoustic parameters from multiple languages when a subword unit is common to two or more 
languages (section 4.3, Multilingual acoustic module, teaches that for phonemes shared by two 
or more languages, the number of hidden units is equal to the number of languages sharing the 
phonemes. 

Regarding claim 2, Riis discloses (Section 3, Multilingual TTP Mapping) accepting a 
plurality of sets of utterances, each set corresponding to a different one of the plurality of 
languages, the utterances in each set being spoken by the native speakers of the language of each 
set; and training a set of acoustic models for the subword units using the accepted sets of 
utterances and pronunciations estimated by the single pronunciation estimator from text 
representations of the fraining utterances. 

Regarding claim 3, Riis discloses a first training word in a first set in the plurality 
corresponds to a first language and a second training word in a second set corresponds to a 
second language, the first and second training words having identical text spellings, the received 
pronunciations for the first and second fraining words being different (Figure 3). 

Regarding claim 4, Riis discloses utterances of the first and the second fraining words are 
used to frain a common subset of subword units (Figure 3). 
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Regarding claim 5, Riis discloses (Section 4.1, TTP Mapping) the single pronunciation 
estimator uses a decision tree to map letters of the text spellings to pronunciation subword units. 

Regarding claim 6, Riis discloses training the single pronunciation estimator further 
comprises: forming, from sequences of letters of each fraining word's textual spelling and the 
corresponding grouping of subword units of the pronunciation, a letter to subword mapping for 
each training word (Section 4.1, TTP Mapping, 4.3, Multilingual acoustic module); and training 
the single pronunciation estimator using the letter-to-subword mappings (section 4). 

Regarding claim 7, Riis discloses fraining the single pronunciation estimator and fraining 
the acoustic models is executed by a nonportable programmable device (section 4). 

Regarding claim 8, Riis discloses generating, for each word in a list of words to be 
recognized, an acoustic word model, the generating comprising generating a grouping of 
subword units representing a pronunciation of the word to be recognized using the single 
pronunciation estimator (section 4.3). 

Regarding claim 9, Riis discloses grouping of subword units is a linear sequence of 
subword units (section 3, Multilingual TTP Mapping to section 4.3, Multilingual acoustic 
module). 

Regarding claim 10, Riis discloses grouping of the acoustic subword models is a linear 
sequence of acoustic subword models (section 4). 

Regarding claim 11, Riis discloses the subword units are phonemes (section 3, 
Multilingual TTP Mapping to section 4.3, Multilingual acoustic module). 
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Regarding claim 12, Riis discloses grouping of subwords is a network, and the network 
represents two pronunciations of a word, the two pronunciations being representative of 
utterances of native speakers of two languages (section 4.3). 

Regarding claim 13, Riis discloses processing an utterance; and scoring matches between 
the processed utterance and the acoustic word models (figure 2). 

Regarding claim 14, Riis discloses generating the acoustic word model, processing the 
utterance, and scoring matches is executed by a portable programmable device (section 1, 
Introduction). 

Regarding claim 15, Riis discloses the portable programmable device is a cellphone 
(section 1, Introduction). 

Regarding claim 16, Riis discloses the utterance is spoken by a native speaker of one of 
the plurality of languages (figure 2; section 4). 

Regarding claim 17, Riis discloses the utterance is spoken by a native speaker of a 
language other than the plurality of languages, the language having similar sounds and similar 
letter to sounds rules as a language from the plurality of languages Figure 2; section 4). 

Regarding claim 18, Riis discloses a method for recognizing words spoken by native 
speakers of multiple languages, the method comprising: generating a set of estimated 
pronunciations, using a single pronunciation estimator, from text spellings of a set of acoustic 
training words, each pronunciation comprising a grouping of subword units, the set of acoustic 
fraining words comprising at least a first word and a second word, the first and second words 
having identical text spelling, the first word having a pronunciation based on utterances of native 
speakers of a first language, the second word having a pronunciation based on utterances of 
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native speakers of a second language (Figure 2; Figure 3; section 3, Multilingual TTP continuing 
to section 4.3, Multilingual acoustic module); mapping sequences of sound associated with 
utterances of each of the acoustic training words against the estimated pronunciation associated 
with each of the acoustic training words (Figure 2; Figure 3; section 3, Multilingual TTP 
continuing to section 4.3, Multilingual acoustic module); using the mapping of sequences of 
sound to estimated pronunciations to generate acoustic subword models for the subword units in 
the grouping of subwords, the acoustic subword model comprising a sound model and a subword 
unit (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual 
acoustic module); and calculating an acoustic subword model for each subword unit, based on 
the pronunciations in the plurality of sets of training words, by mixing distributions of acoustic 
parameters from multiple languages when a subword unit is common to two or more languages 
(section 4.3, Multilingual acoustic module, teaches that for phonemes shared by two or more 
languages, the number of hidden units is equal to the number of languages sharing the phonemes. 

Regarding claim 19, Riis discloses a method for multilingual speech recognition 
comprising: accepting a recognition vocabulary that includes words from multiple languages 
(Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual acoustic 
module); determining a pronunciation of each of the words in the recognition vocabulary using a 
pronunciation estimator that is common to the multiple languages (Figure 2; Figure 3; section 3, 
Multilingual TTP continuing to section 4.3, Multilingual acoustic module); and configuring a 
speech recognizer using the determined pronunciations of the words in the recognition 
vocabulary (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, 
Multilingual acoustic module); and calculating an acoustic subword model for each subword 
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unit, based on the pronunciations in the plurality of sets of training words, by mixing 
distributions of acoustic parameters from multiple languages when a subword unit is common to 
two or more languages (section 4.3, Multilingual acoustic module, teaches that for phonemes 
shared by two or more languages, the number of hidden units is equal to the number of languages 
sharing the phonemes. 

Regarding claim 20, Riis discloses accepting a training vocabulary that comprises words 
from multiple languages (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 
4.3, Multilingual acoustic module); determining a pronunciation of each of the words in the 
training vocabulary using the pronunciation estimator that is common to the multiple languages 
(Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual acoustic 
module); configuring the speech recognizer using parameters estimated using the determined 
pronunciations of the words in the training vocabulary; and recognizing utterances using the 
configured speech recognizer (Figure 2; Figure 3; section 3, Multilingual TTP continuing to 
section 4.3, Multilingual acoustic module). 

Regarding claims 21-46, the computer implementation of Riis multilingual system 
provides support for a computer program product and apparatus for realizing the multilingual 
speech recognition acoustic models. The claim limitations recited in 21-46 to cause the 
processing of multilingual speech recognition are similar in scope and content to the method 
claims of 1-20, and are therefore rejected under similar rationale. 
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Response to Arguments 

Applicant's arguments with respect to claims 1-46 have been considered but are moot in 
view of the new ground(s) of rejection. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANGELA A. ARMSTRONG whose telephone number is 
(571)272-7598. The examiner can normally be reached on Monday-Thursday 1 1 :30-8:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick N. Edouard can be reached on 571-272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Elecfronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Angela A Armstrong/ 

Primary Examiner, Art Unit 2626 



